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ABSTRACT 



Every administrator bears primary responsibility for 
planning the accomplishment of those objectives and activities that 
fall in his area. When responsibility is relatively restricted, the 
informal and intuitive methods that constitute "sound judgment” 
suffice. However, as his area of responsibility widens to include 
increasingly complex tasks more formal methods of planning and 
control may be needed. As, for example, when planning for or 
controlling whole systems of activities (such as a school district), 
when the activities are new or experimental, or when large sums of 
money are to be expended. A vast literature exists on methods of 
planning for systems, for 'Comprehensive planning* 11 Many of the ideas 
and methods found in the contemporary literature are discussed and 
evaluated in the first part of this report. An extensive annotated 
bibliography follows, with five main sections covering planning 
process information; system-wide planning methods; system 
characteristics and qualities; politics, community, implementation 
and communications; and, finally, other relevant bibliographies. The 
report concludes with author and subject indexes. (Author/GS) 
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PREFACE 



Every administrator bears a primary responsibility for planning the 
accomplishment of those objectives and activities that fall in his area 
of responsibility. When responsibility is relatively restricted in extent, an 
administrator can well use the informal and intuitive methods that consti- 
tute T, sound judgement;" however, as his area of responsibility widens to 
include tasks of increasing complexity, he may feel that it would be useful 
to implement more formal methods of planning and control. When the admin- 
istrator is responsible for planning or controlling whole systems of 
activities (such as a school district) or when the activities are themselves 
new or experimental or when large sums of money are to be expended, then the 
use of one or another formal planning process may become necessary in order 
to maintain the desired degree of control. 

A vast literature exists on methods of planning for systems, for 
"comprehensive planning." It is hoped that those who are involved in system- 
wide planning will find something of use to them in this publication. 

The first section discusses and evaluates many cf the ideas and methods 
that are to be found in the contemporary literature on comprehensive planning. 
Since, in this young field of study, accepted standards for evaluation of 
various ideas and methods do not exist, it must be confessed that the author’s 
subjective judgements have of necessity influenced the presentation. 

An annotated bibliography constitutes the second section. The items 
there have been selected by the author for their potential usefulness to those 
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responsible for comprehensive planning. Often an abstract, prepared by the 
author of an item, was available. In those cases the author r s abstract was 
accepted as an annotation. Credit for the remaining annotations belongs to 
Lewis Polin. Joan Rosenstein and Elenore Pritchard provided editorial 
assistance. Loretta Margulies and Shakuntala Asnani handled typing and 
logistics for the project in their usual efficient manner. 



Sanford Temkin 
September, 1970 
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ORGANIZATION AND USE OF THIS BIBLIOGRAPH v 



The arrangement of this bibliography is classified into five general 
categories. The categories are 1.0, PLANNING PROCESS INFORMATION; 2.0, 

SYSTEM WIDE PLANNING METHODS; 3.0, SYSTEM CHARACTERISTICS AND QUALITIES; 

4.0, POLITICS, COMMUNITY, IMPLEMENTATION AND COMMUNICATION; 5.0, BIBLIOGRAPHY. 
Each category is subdivided, e.g., 4.1 is Politics and Community and 4.2 
is Politics - Implementation. Within each subdivision the references are 
listed alphabetically by author. Author and subject indexes follow the 
annotated bibliography section. The author index refers the reader to the 
appropriate page number in the annotated bibliography section. The subject 
index indicates the item number where the subject can be found, e. g., 

4.1.2 indexes the second article of subdivision 4.1. 



Fleur Weinberg, Librarian 
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AN EVALUATION OF THE LITERATURE 
OF COMPREHENSIVE PLANNING 



The term "comprehensive 11 has meanings which are easily confused. On the 
one hand, it is always possible to include more information in the planning 
process and thus label the process as more comprehensive. On the other hand, 
the word, may be employed in a completely different sense. But before con- 
sidering this other meaning it is necessary to define the term "system. " A 
system may be defined as a set of mutually dependent activities which is 
directed towards a set of objectives common to all those activities. The 
term comprehensive in its second sense refers to a planning process which 
determines and evaluates alternative decisions within the total context of 
the set of shared objectives. As used in this monograph the term comprehen- 
sive, then, does not denote a concern for more information about a school 
district per se, but a concern for that information which allows one to view 
a district's operations as being directed toward a set of objectives common 
to all the activities of the district. 

The indispens ible elements of a process are called "necessary and suf- 
ficient conditions." Planning processes have indispensible types of 
information. The nature and specific interrelationships among these types 
of information depend upon what the planning process is designed to achieve. 
There are several ways to attempt to identify the types of data that are 
indispensible for a comprehensive planning process. The first approach 
selected by this writer was to use mathematics to select the types of data 
necessary to achieve the best possible fulfillment of all the objectives of 
the system, given the restraints imposed by the resources available. Another 

o 
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way tc identify ind ispens ible types of information would be to read compre- 
hensive planning literature in the hope that they could be recognized 
empirically and intuitively. Yet another we T would be to consult with 
experienced planners and planning theoreticians. In actuality, all three 
approaches influenced this paper. The types of data that are needed tor a 
comprehensive planning process are: 

1. A sec of planning objectives (or overall objectives) of the whole 
system. 

2. Information about the relative importance ( M weight' r ) that various 
people involved in the system believe is appropriate for the 
different planning objectives. 

3. At least one indicator of how the degree of attainment of each 
objective is to be judged (a ,T perf ormance indicator 0 ). 

4. Information about the potential importance of various programs and 
activities to each of the planning objectives (an approximation to 
the concept of the "production function”). 

3. Performance outcomes for each performance indicator. 

6. Costs. 

There are two considerations that arise when one discusses these types of 
data. First there are questions about measurement and data collection 
techniques. Second there are questions about relationships among types of 
data, i.e., the synthesis, or how to "put it all together." Information types 
are discussed in these two regards. References are made to particular 
bibliographical items that are annoL?ted in the main body of this monograph. 
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Page numbers refer to the reference's page number in the annotated bibiiogra 
phy . 

Planning o bj ectives or overall object i ves 

If one visualizes a line that has vague goals at the extreme left and 
behavioral objectives at the extreme right, planning objectives arc found 
"about 2/10" of the way from the left extreme (see sketch). Planning 
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Behavioral 

Objectives 



objectives are a higher order abstraction in relation to very specific 
instructional objectives and as such play a different role in planning. 
Planning object ives have a single function - to allow t ic* operations of the 
school district to be placed In a network of relationships for subsequent 
planning and discussion. This approach contracts with one that is somewhat 
widely held - nainely that objectives to be useful must b»2 measurable. It 
is true that measurement is important; but, measurement should be reserved 
for the criteria of performance, which have the functicn of determining the 
extent to which a given activity has done what it was intended to do for the 
system as a whole. 

The Downev study (p. 16) has the mo* t to say about goals and objectives 
the Educational Testing Service study (p. 15) > prepared for the Pennsylvania 
Department of Education, has generated a great deal of performance data for 
"ten goals of education" since 1965; and the Radnor paper (p. 16) represents 
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a recent statement by a small subc^ban district that is interested in objec- 
tives, performances, and planning for change. 

Assignment of weights to planning objectives 

As one reads about decision-making and listens to decision-makers, a 
clear message emerges - priorities are all important in planning. This writer 
must disagree with certain implications of this conclusion. Deciding on 
priorities may be important to "get a feel for things" but they are not all 
important in the planning process. For instance, picture a decision-maker 
with three objectives which have been ranked in terms of their priorities. 

Does he allocate resources exclusively to the highest priority objective? 

Tf not, how much does he allocate to the highest priority objective before 
he allocates resources to the middle priority objective? Or does he allocate 
some resources to each objective? If so, how? 

These questions can be answered partially by an assignment of relative 
weights to the planning objectives. Once this notion is accepted then a host 
of other questions emerge. Some of these are: 

1. How can weights be obtained from people? 

2. Can weights be obtained from both individuals and groups? 

3. Are these weights really measurements? 

4. How do the weights obtained from one person compare to those obtained 
from another? 

There are three groups of method? for obtaining assignment of relauive 
importance or preference from people. The Churchman-Ackof f Method may be 
applied to individuals or groups. The metrod forces the subject to consider 
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and reconsider the relative importance of objectives singularly and in 
combination. Hopefully, the subject is able to generate a set of internally 
consistant assignments of importance to ea^h objective. This method and 
variations of it ate common in the literature - see, for instance, Churchman, 
Ackoff and Arnoff (p . 18) *, and Stimson (p . 22)* 

The Neumann-Morgenstern Cardinal Utility approach has been used primarily 
by economists and statisticians. It centers on an individual's responses 
to selected proposals. These proposals are usually stated in terms of dollar 
return and probability cf accomplishment. A good description of the method 
can be found in Baumol (p. 18), while a worthwhile empirical study of 
individual utility functions can be found in Graysons classic study p . ^0) 

of decision-making practices in th 2 oil-well drilling industry. 

The Delphi Method first appeared in the literature around 1963. Essen- 
tially, it is designed for group judgements and arrives at consensus by using 
a series of steps. In each step subjects are told the results of judgements 
made at the end of the previous step. Those people who maintain extreme 
judgements justify their positions for others to evaluate. By this procedure 
consensus is ultimately achieved. Although this method is currently extremely 
popular very little had been done to explore its assumptions until recently. 
Norman Dalkey, one of the Delphi innovators, recently has discussed some of 

these assumptions (p. 18), while Rescher (p. 21) provides additional insight 

into the subtleties of the method. 

Since judgements of relative importance (weights) obtained from one 
person cannot be added or subtracted from those obtained from another, inter- 
personnel utility comparisons cannot be handled directly. The best solution 
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would seem to be that of avoiding summing individual utility functions. 
Accordingly, any evaluation of alternatives for purposes of allocating 
resources would be done for a specific set of weighted planning objectives. 
This would mean that each time a set of preferences is introduced a new 
allocation of resources would result. When the preferences of many indi- 
viduals or groups are to be explic'- j considered, computer processing is 
undoubtedly required. 

The Churchman-Ackof f Method claims to meet the basic additivity and 
transitivity assumptions for measurement. Bvumol indicates Neumann- 
Morgenstern indexes also meet these requirements. As was indh:ated earlier 
there is still much to be learned about the Delphi Method. 

This author feels that measurement theory and methodology lag far 
behind the need for practical real-world methods 

Another domain which may be addressed with regard to preference assign- 
ments is that of pupil needs, an area in which little has been done. 

Standard economic theory indicates that the best dintr ibution of budgetary 
funds among all activities occurs when the additional usefulness of each 
dollar for an individual are proportional to the prices he pays. These, 
so-called, "marginal utilities" are assumed to is present a situation in 
which the individual’s needs are satisfied. School districts also have a 
budget to allocate and it would seem reasonable to take into consideration 
pupil needs. As one examines the literature there is frequent mention of 
"meeting the needs of kids" but there is very little to guide someone on 
how to assess these needs. In this author’s opinion the most sensible 
approach, is the study by Campbel. and Markle at the Far West Laboratory 
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for Educational Research and Development (p. 13) and Mirsky’s work on :i pupil- 
event analysis'" (p. 14). 

Hopefully, data on pupil needs would serve as input into the process by 
which assignments of relative importance to overall objectives would be deter- 
mined and represented. It is argued by this author, that perhaps a meaningful 
arrangement can emerge as various community groups discuss and debate the rel- 
ative importance of different pupil needs and their implications for the school 
district's planning objectives. 

• erf rrn nce criteria or ind ^ators 

Just as there are many ways to think about objectives, one encounters many 
ways to assess performance. A generalized understanding of performance criteria, 
in this author T r experience, results from the notion of '"operational criteria. ,T 
Although operational criteria are arbitrary and subjective, they explicitly state 
what will be observed, by whom* when, and under what conditions. Behavioral 
criteria, a sub -class of operational criteria, are designed for observing behav- 
ior and are most useful for assessing behavioral objectives. Assessment of per- 
formance for planning objectives may or may not necessitate direct behavioral 
observation. 

This author partitions performance assessment into four steps: 

The decision-maker must specify a performance criterion. A performance 
criterion specifies what will be observed to determine the extent to 
which a given objective has been met. 

It is determined who will do the observing, when and under what 
condi tions . 

Possible performance outcomes are anticipated and their merit inter- 
preted. Mathematically, we can say, an X-axis would display various 
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outcomes and a Y-axis would show the respective values as a 
percentage (0 - 100) . 

4. The final step involves making the observations and using the index 
to interpret the merit (or value) of the outcome that did result. 

The basic idea underlying a performance indicator is that a knowledgeable 
decision-maker is able to specify, given some assistance and support, wnat 
information he would like to evaluate in order to determine the extent to 
which a given overall objective has been served. An interesting character- 
istic of a performance indicator, defined in the manner suggested, is that 
two children who receive the same item score (each get four correct out of a 
total of eight responses) may be credited with different performance index 
outcomes, depending on the assigned importance of that score to each indi- 
vidual child r s progress. 

Effectiveness 

Concepts of effectiveness have been applied in various ways in planning. 
Usually the term is used as synonymous with good performance. A few authors 
indicate that effectiveness may be viewed as a function of two variables — 
performance and the potential of the respective activities to contribute to 
the overall organization or system. An assessment of the ongoing system, as 
it is operating, can be made by considering effectiveness as a function of 
performance and worth. The requirements tor such an evaluation are: 

1. Each overall objective is assigned a weight in accord with its 
importance relative to the other overall objectives. 

2. Each activity conducted under the auspices of the ongoing system 
is assigned a weight in accord with its relative importance or 
potential to contribute to its overall object ive(s) . This allows 
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the relative worth of each overall objective to be partitioned among its 
component activities. 

3. A performance index, which will give a more or less formalized indica- 
tion of the extent to which the activity has done what it could do, has 
been designed for each activity. 

When acti\ity performance is perfect (1.0, or 100% in other words, on the 
performance index) the full worth of the activity is realized and it is said to 
be totally effective. The effectiveness of an activity, in this context, is its 
performance index outcome weighted by its potential for contributing to the sys- 
tem. This implies that if two activities perform equally, the one with the 
higher potential value, from the point of view of a system-wide evaluation, is 
more effective. 

Authors who give considerable attention to the synthesis of concepts dealing 
with objectives, values and performance are Fallon (p . 19), Chuang and Peper 
(p. 30), and Temkin (p. 31). 

Most authors accept the notion that output of a system should in some sense 
be related to input so that alternatives can be evaluated. One family of 
such methods comes from the "input-output" methods of Wassily Leontief.* The 
shortcomings of such applications are that they avoid consideration of 
the process by which different sets or inputs relate to different outputs. 

In applications to education, process variables are generally mixed with 
input variables and those related to characteristics of the child and the 
home. But were the variables unconfounded there would still remain the problem 
of relating input-to-process-corrbinations to process-to-output-combinations . 

*W. W. Leontief, The Structure of American Economy , 1919-1939, New York, 
1931, 2nd edition. 

o 
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One of the most interesting methods belongs zo Dyer (p. 43), who suggests 
a scheme that combines input-output search for hypotheses with experi- 
mental design considerations. 'Cere it not for a scarcity of funds for 
education, Dyer r s method could make a practical contribution to educational 
decision-making, since it would have the capability of taking educational 
processes into consideration. 

Another way to bring input, process and output information together 
would be to develop a series of curves, one for each activity, that relate 
effectiveness to cost. This information would allow one to e/aludte the 
amount of change in effectiveness for each change in cost for an ongoing 
activity. This could be plotted as a cost-effectiveness curve. Proposed 
alternatives could be introduced by developing a new curve and recognizing 
that new activities often have associated new fixed costs. Once the curves 
are drawn, then mathematical methods such as linear programming or a non- 
linear adaptation (see McNamara, p. 39 or Sisson's comments in Temkin's 
Appendix, p. 42) could be used to search for preferred alternatives. 

Synthesis and implementation 

PPBS-type methods were originally designed for research and development 
organizations. This planning mechanism was used to help select from among 
alternative ways to produce a given capability. The process generally began 
by defining a set of capabilities or "missions. M This was followed by the 
development of alternative programs , td budgets that could support the 
respective alternatives. Generally a multi-year plan resulted. 

One of the major needs in managing a school system is a technique 
capable of relating outputs to costs. This need is dramatized by the 
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increasing demands on a small supply of educational dollars. 

A PPBS is such an approach. The method, however, is more applicable 
to the planning of a completely new school system than to planning changes 
for an existing school system 

A fundamental problem in applying PPBS to schools is that it does not 
lend itself to orderly transition from where a school district is to vhere 
management wants it to be. Orderly systemic change is a dynamic systems 
engineering problem that does not lend itself to a situation in which ends 
are merely specified and necessary changes are, assumed to just happen 
automatically once this is done. People are responsible for conducting the 
affairs of schools. These people are individuals whose present behaviors 
are based on attitudes, values, beliefs, customs, training and experiences. 
Croston and Gregory (p . 60) indicate that the usual view of an organization, 
from within, supports a situation in which necessary information tends to flow 
upward:; and downwards, but if planning information is to be related to 
organizational objectives a horizontal flow of information is needed. Helping 
people within the organization to understand and become comfortable with 
organi zat ional objectives and horizontal communication flows is a necessary 
condition for comprehensive planning. Other necessary conditions are 
executive commitmen; (see Benton and Tenzer p. 59^; widespread, meaningful 
involvement in planning and decision-making (see Eberle p. 61); sometimes, 
organizational restructuring in order to facilitate planning and change 
(see Andes and Kimbrough p. 51 > and Maguire p. 56); and a good management 
system once intended changes are specified (see Cook p. 60). 
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The essence of a comprehensive planning process for schools, in the 
opinion of this writer, is to show teachers how to look at their work 
in the classroom so that they can evaluate what they do. In addition, roles 
are to be defined and clarified for principals, central office administrators 
and community groups. Effective and efficient change seldom comes about 
by means of changes in the accounting system or the budget alone. 

In an ongoing, dynamic, and changing system the superintendent can 
detect changes in pupil achievements and changes in program budgets. 

But the simple fact remains that meaningful change can be best accomplished 
by developing and maintaining an educational process that is responsive 
to changing needs. 

The above mentioned methods, techniques, and ideas can, and already 
have, contributed, in various measure and with varying results, to the. 
problem of comprehensive system-wide planning in education. The 
following bibliography represents, hopefully, the vast emerging literature 
on the subject. 
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PLANNING PROCESS INFORMATION 



1 . 1 Pupil-Needs 

1) Campbell, Vincent N., and David G. Markle. " Identifying and 

Formulating Educational Problems." Berkley, Calif.: 

Far West Laboratory for Educational Research and Development, 
April, 1968. 

"The aim of this project was to develop effective techniques for 
identifying educational needs and formulating them into well defined 
problems. Tne critical incident technique was used to identify need 
symptoms perceived by educators for a representative sample of schools 
throughout the region served by the Far West Laboratory. A hierarchy 
of categories was developed to describe these need data, which wert 
then classified and tabulated by category. The problem formulation 
techniques explored informally included interview, observation, and 
small group discussion. We decided that knowledge of problem defin- 
ing behavior is too primitive to justify step-by-step sys temi zation 
of the process of formulation. The techniques we developed rather 
relied heavily on intuitive processes, and systematic control was 
aimed toward insuring that the product of the formulation contained 
the following elements: outcomes desired, values underlying outcomes, 
kinds of evidence for outcomes, present conditions, solution 
possibilities and immediate action alternatives. We experimentally 
compared three group problem defining techniques which differed on 
two dimensions: (a) whether or not the formulation was structured to 
obtain the elements listed above, and (b) whether or not the inquiry 
was directed by an experienced outside formulator. Eighteen three- 
person groups of educators were used in all. The problem definition 
produced by each group was evaluated independently by two other edu- 
cators from the same district. Results favored the directed groups; 
among the undirected groups completely unstructured groups rated as 
high or higher than groups producing a structured definition. 
Implications for training and promising directions for further work 
on problem formulation systems are discussed." (Author) 

2) McElhinney, James H., Richard C. Kunkel, and Lawrance A. Lucas. 

"Evidencer of School Related Alienation in Elementary 
School Pupils." Paper presented at the American Educational 
Research Association, Minneapolis, March, 1970> 

The students were given a 72 item questionnaire in which 11 items 
included responses that might suggest alienation. The authors 
provide an analysis of th^ results of each of the 11 items which 
shows that various degrees of alienation exist in every school 
included in the sample. Finally, the authors suggest implications 
of these findings to alleviate th’se problems. 
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3) Mirsky, Joseph. "Pupil-Event Analysis for Needs Assessment." 

Philadelphia: Research for Better Schools, December, 1969. 

The author presents a brief argument foi sampling "pupil-events" 
in order to learn about the nature of pupil-needs. He suggests 
that pupil-need assessments examine the functions of the school 
system as they relate to the things students do. 

4) Popham, James W. "Educational Needs Assessment in the Cognitive, 

Affective, and Psychomotor Domain. M A presentation at three 
ESEA Title III Regional Workshops sponsored by the U . S. Office 
of Education, Ft. Lauderdale, Fla., November, 1969; 

Philadelphia, Penna., December, 1969; Scottsdale, Ariz., 

December, 1969. 

A first problem, identifying educational preferences, can be 
remedied by utilizing the behavioral objectives which are being 
developed by the Instructional Objectives Eychange of U.C.L.A. 
and then sampling the clientele to determine the priority of 
the objectives. A second problem, the identification of the 
learner's status, i.e., level if attainment can be solved by 
using criterion-referenced tests based upon the behavioral 
objectives. The third problem involves a contrast of preferences 
with status, i.e., a comparison of desired outcomes to current 
learner status, can be solved through the administration of 
criterion referenced tests. 

(See also the annotation of Marvin Alkin's article, "Behavior 
Objective Specifications in Evaluation: Relevant or Irrelevant" 
which appears in section 1.4). 

3) Rhea, Buford, ejt a_l. "Measures of Child Involvement and Alienation 
from the School Program." Chestnut Hill, Mass.: Boston College, 
1966. 

The purpose of this report is to assess the alienation and/or involve- 
ment of high school students through the use of short interviews sup- 
plemented by questionnaires. The sample consisted of two highly-rated, 
suburban Boston schools whose students were oi middle to upper-middle 
class backgrounds. Although the authors concluded that students 
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did not display che traditional symptoms of alienation, they were 
not able to evaluate a disturbing anomaly which resulted in stu- 
dents converting intended educational goals into spurious means (good 
grades, admission to college, notion of "preparation") . They feel 
that more research should be conducted to evaluate how a "task- 
orj entat ion" view of schools affects studenc alienation. Further- 
more, the authors characterize a pattern that has developed in the 
two schools as "institutional paternalism.' In the final sections 
of the report, the writers discuss some of i:he conceptual problems 
associated with measures of alienation and Involvement. 



6) Woodbury, Charles A. , e^t al . "Research Mod'll for State Educational 
Needs Assessment." Paper presented at the Annual Meeting of the 
American Educational Research Association, Minneaoolis , March, 
1970. 

Th:.s paper summarizes the Virginia Needs Assessment Project’s 
research strategy in determining learner-c r^ented needs in the cog- 
nitive, affective and psycho-motor domains. This approach requires 
the delineation of goals (four are mentioned), programma t lc efforts 
(defined as implemented goals) , and programmatic outcomes (measured 
by scores on standardized tests and scores on affective rating 
scales). Needs are represented as gaps that exist between the 
goals and the outcomes. Also included in The report are subordinate 
assessment strategies concerning the affective domain and goal 
percept ions . 



1 . 2 Overal . Objectives 

A Fl an for Evaluating the Quality of E duca tional Pro g rams in 
Pe nnsylv a nia : Volumes I, II, and 111 , Princeton, N . J . : 
id national Testing Service, June, 1^6 5 . 

lu tiai*y, the report lists and discusst ! ten educational goals 
fo; Pennsylvania schools. Included is a proposed survey of 
public opinion to assess Li a relative importance of these goals. 

A method logy is outlined f ^ the development of performance 
indicators which eventually »ould result :.n inter-school comparisons. 
Thiise ^ valuations would point to needed educational policy adjust- 
ments for schools with comparable inputs and surrounding conditions. 

Note: The measures designed as a result i: this study have been 
used in Pennsylvania by the Department of Education in what has 
been referred to as "The Quality of Education Study." 

This annotation also appears in section 1.4. 
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